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EXECUTIVE  SUMMARY 


\£i  7-„ 

A  small,  wearable  high  resolution  display  called  "Private  Eye"  was 
evaluated . ""'This  monochrome  (red)  display,  designed  to  be  worn  in  front  of 
one  eye,  measures  about  1"  x  1"  x  3"  and  weighs  2  ounces.  .Using 
conventional  semiconductor  and  optical  techniques ,  it  provides  the 
equivalent  amount  of  information  to  that  of  a  12 -inch  monitor  driven  by  a 
PC. 


"■  A  highly  concentrated,  data  input  task  was  chosen  to  obtain  hands-on 
experience  with  this  new  display  device,  and  to  quickly  identify  the 
potential  advantages  and  deficiencies  in  the  device.  This  task  required 
the  keyboard  entry  of  information  from  the  Private  Eye  into  simulated  air 
traffic  flight  strips,  with  key  entries  displayed  by  a  personal  computer. 
Subjective  evaluations  were  the  emphasis  of  the  study,  and  were  quantified 
on  a  seven-point  rating  scale.  Data  entry  rates  and  errors  were  also 
measured.  Seventeen  subjects  were  used,  with  a  wide  range  of 
characteristics  in  terms  of  level  of  experience,  corrected  visual  deficits, 
and  keyboard  skills.  To  obtain  a  performance  baseline,  the  flight  strips 
were  entered  from  printed  material  as  well  as  from  the  Private  Eye. 

The  average  ratings  for  the  subjective  evaluation  are  shown  below.  A 
rating  of  1  or  2  is  considered  favorable,  3  to  5  is  neutral,  and  7  or  8  is 
unfavorable . 


Hi 


Generally  liked  device:  3.2 
Helped/hurt  ability  to  enter  data:  3.4 

Ability  to  move  attention  between  Private  Eye  and  PC  monitor:  3.0 
Ability  to  move  attention  between  Private  Eye  and  keyboard:  2.6 
Color  of  Private  Eye  characters:  2.8 
Legibility  of  Private  Eye  characters:  1.9 
Fatigue  during/after  task:  2.3 

Perceived  display  flicker  (4  of  17  subjects):  3.5 
Comfort  of  device:  2.9 

Thus  the  ratings  of  the  Private  Eye  generally  ranged  from  slightly 
favorable  to  neutral,  perhaps  affected  by  a  "built  in"  bias  toward  mid¬ 
range  ratings.  The  subjects  had  no  difficulty  with  depth  perception, 
although  the  device  blocked  a  large  portion  of  one  eye's  visual  field.  The 
red  color  of  the  display  and  its  brightness  and  contrast  likewise  presented 
no  problems  to  the  subjects.  The  rather  concentrated  but  brief  use  of  the 
Private  Eye  apparently  caused  no  appreciable  eye  fatigue;  however,  some 
minor  discomfort  was  reported  from  the  head  harness,  which  the  vendor  is 
redesigning. 

The  character  font  employed  in  the  Private  Eye  apparently  caused 
confusion  between  certain  characters.  One  of  the  subjects  said  he  was 
unable  to  consistently  see  the  entire  Private  Eye  display  and  two  others 
had  difficulties  when  using  bifocal  glasses.  These  problems  might  have 
been  corrected  by  careful  repositioning  or  refocusing  of  the  Private  Eye. 
Five  subjects  also  reported  a  transient  "ghosting",  flickering,  or  movement 
of  the  image,  which  might  be  explained  by  head  or  eye  movement. 

The  differences  were  not  significant  when  comparing  keyboard 
performance  with  the  Private  Eye  to  performance  with  the  paper  material. 
Using  the  Private  Eye,  the  subjects  entered  data  at  a  rate  of  76.6 
characters/minute  with  a  grand  total  of  95  errors  made  by  all  subjects; 
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using  the  paper  material  they  entered  at  80.1  characters/minute  with  a 
grand  total  of  70  errors.  There  appeared  to  be  no  consistent  relationship 
between  the  subjects'  individual  characteristics  and  their  ratings  of  the 
Private  Eye. 

As  a  general  conclusion,  this  evaluation  identified  no  intrinsic 
limitations  in  the  usability  of  the  Private  Eye  device,  although  it  is  not 
the  optimum  display  for  a  continuous  entry  task.  Used  in  conjunction  with 
an  appropriate  input  device,  it  could  have  significant  advantages  in 
applications  where  portability,  privacy,  and/or  a  "personal"  window  of  a 
larger  display  is  required.  D63  is  in  the  process  of  obtaining  an  updated 
model  of  the  Private  Eye.  Combining  the  Private  Eye  with  live  input 
devices,  such  as  voice  recognition  or  head  trackers,  may  provide  for  hand- 
free  portable  operations  to  support  future  field  maintenance  or  virtual 
command  post  requirements. 
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SECTION  1 


INTRODUCTION 

This  report  describes  a  preliminary  evaluation  of  a  small, 
monochromatic  wearable  device  called  the  Private  Eye.  This  miniature 
display,  designed  to  be  worn  in  front  of  one  eye,  can  provide  an  equivalent 
amount  of  information  to  the  full  screen  of  a  standard  12 -inch  monitor 
driven  by  a  PC.  Figure  1-1  shows  a  subject  wearing  the  Private  Eye  seated 
in  front  of  a  personal  computer  (PC),  the  configuration  used  in  this 
evaluation.  (Note:  the  Private  Eye  in  this  illustration  is  positioned 
further  from  the  subject's  eye  than  the  normal  distance  of  about  an  inch.) 

The  evaluation  provided  "hands-on"  experience  with  a  prototype  of  this 
device,  which  is  in  the  process  of  entering  the  commercial  marketplace. 
Subjects  were  given  a  brief  task  designed  to  require  the  repeated, 
continuous  use  of  the  displayed  information,  immersing  them  in  the  use  of 
the  Private  Eye.  The  task  was  not  chosen  because  it  was  an  optimum 
application  of  the  Private  Eye.  In  fact,  a  highly  concentrated,  "worst 
case"  keyboard  interaction  task  was  chosen,  so  that  any  perceived  flaws  in 
the  device  would  become  readily  apparent.  Thus  the  subject's  task  was  to 
type  information  displayed  on  the  Private  Eye,  a  task  requiring  the 
concentrated  use  of  this  device  in  conjunction  with  a  standard  keyboard  and 
a  PC  monitor.  Subjects  were  given  little  time  or  opportunity  to  become 
accustomed  to  using  the  device,  again,  a  worst-case  condition. 

Subjective  evaluations  of  the  Private  Eye  were  the  emphasis  of  this 
study.  The  subjects'  opinions  of  the  device  were  recorded,  and  were 
quantified  via  rating  scales.  Performance  was  also  quantified  in  terms  of 
typing  rate  and  errors. 
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Figure  1-1.  Flight  Strip  Dat 


The  evaluation  was  structured  to  provide  meaningful  results  within  a 
short  time,  with  a  minimum  amount  of  software  and  facility  development,  and 
using  a  variety  of  different  individuals  as  subjects.  Although  its 
applicability  is  not  restricted  to  a  particular  project,  the  evaluation  was 
conducted  within  the  context  of  an  air  traffic  control  task,  namely  the 
entry  of  flight  strip  data.  In  current  air  traffic  control  operations, 
these  flight  strips  are  manually  prepared  and  maintained.  The  automated 
entry  and  maintenance  of  the  strips  thus  is  germane  to  the  New  Mobile 
RAPCON  project. 

Flight  strip  information  was  entered  from  printed  material  as  well  as 
from  the  Private  Eye.  Since  it  is  likely  that  data  entry  would  be  faster 
and  less  error  prone  using  the  familiar  paper  material  (especially  for  more 
skilled  typists),  there  was  no  intent  to  directly  compare  the  Private  Eye 
to  the  printed  material.  Since  the  subject's  relative  keyboard  skills  were 
of  primary  interest,  performance  on  printed  material  was  intended  to  form 
only  a  general  baseline  for  the  Private  Eye  data  entry  task.  The  order  of 
presentation  of  the  printed  material  and  the  Private  Eye  material  was 
counterbalanced  to  counteract  some  of  the  effects  of  learning. 

The  Private  Eye  vendor,  Reflection  Technology  (Waltham,  MA) ,  has 
indicated  that  certain  improvements  to  the  evaluated  prototype  are 
currently  being  designed  (e.g.,  the  harness  which  holds  the  device  on  the 
viewer's  head).  It  is  likely  that  some  of  the  shortfalls  identified  during 
this  evaluation  will  be  addressed  in  the  enhanced  version. 

The  following  sections  of  this  report  describe  the  Private  Eye  and  the 
system  used  for  its  evaluation;  the  detailed  procedures  used;  the  subjects' 
characteristics ;  the  specific  results  in  terms  of  the  subjects'  ratings, 
opinion,  typing  rates,  and  errors;  and  a  discussion  and  interpretation  of 
these  results . 
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SECTION  2 


EQUIPMENT 


The  Private  Eye  uses  a  proprietary  display  technology  based  on 
conventional  semiconductors  and  optical  techniques  to  create  an  image  of  a 
12 -inch  monitor  in  a  miniature  package.  It  measures  1.1"  x  1.2"  x  3.2"  and 
weighs  2  ounces.  The  image  appears  to  be  approximately  18  inches  from  the 
viewer's  eye.  Emissive  display  elements  are  used  for  the  fast  responding 
high  resolution  (720  x  280  pixels),  monochrome  (red)  image.  Because  a 
black  background  is  used,  high  contrast  ratios  are  achieved.  Both  text  and 
graphic  information  can  be  displayed;  however,  in  this  evaluation,  only 
text  data  was  displayed.  See  table  2-1  for  a  listing  of  the  Private  Eye 
specifications . 

A  80386 -based  PC  with  a  VGA  monitor  was  used  to  support  the  display  of 
information  on  this  prototype  Private  Eye,  to  accept  the  subjects'  inputs, 
and  to  store  their  performance  data.  The  Private  Eye  was  connected  to  the 
PC  through  a  4-foot  wire  cord.  Figure  2-1  is  a  block  diagram  of  the 
configuration  used  in  this  evaluation. 

Font  design  was  an  important  aspect  of  working  with  this  prototype. 
Before  beginning  this  evaluation,  it  was  observed  that  standard  PC  monitor 
text  when  displayed  on  the  Private  Eye  was  too  small.  Reflection 
Technology  provided  a  set  of  software  to  allow  a  new  font  to  be  designed  on 
a  Macintosh  computer  using  the  standard  "Superpaint"  software  package. 

Once  built,  using  "Superpaint,"  the  formatted  displays  were  transferred  to 
the  PC  via  a  LAN  at  the  Bedford  Computer  Center.  In  addition,  the 
"Superpaint"  displays  required  90°  rotation,  since  the  prototype  Private 
Eye  image  plane  is  orthogonal  to  the  PC  bit  plane.  Attempting  to  optimize 
the  font,  several  formats  were  examined  before  18  point,  proportionately 
spaced  Times  font  was  selected  for  use.  To  further  minimize  the  effects  of 
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Table  2-1.  Private  Eye  Specification 


Resolution: 
Image  color: 
Image  size: 

Size : 


720  x  280  pixels  standard 

High  contrast  red  pixels  on  black  background 

22  degrees  horizontal  by  14  degrees  vertical.  Equivalent 
to  viewing  a  screen  9.25  inches  wide  by  6.0  inches  high, 
which  is  located  24  inches  away. 

1.1  x  1.2  x  3.2  inches 


Weight : 

Exit  pupil: 

Focus  range: 
Refresh  rate: 
Update  rate: 

Power : 


2  ounces 

0.4  inches  square  at  0.8  inches  from  the  face  of  the  case 
(location  of  user's  eye  to  view  entire  image) 

10  inches  to  infinity  (user  adjustable) 

Approximately  50  hertz 

Display  can  change  images  at  the  refresh  rate.  Choice  of 
interface  may  limit  speed. 

5  volts  DC.  Average  power  under  1/2  watt.  Idle  power 
when  no  pixels  are  illuminated  is  negligible. 
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Figure  2-1.  Evaluation  System  Configuration 


the  font  design,  neither  the  letter  "0"  nor  the  zero  was  used.  Appendix  A 
contains  the  data  that  was  entered  during  the  evaluation  and  is  printed  in 
the  Times  font  as  presented  on  the  Macintosh  computer. 

The  evaluation  was  run  in  a  relatively  quiet  room  with  minimum 
distractions.  The  ambient  light  was  subdued  and  was  from  the  subject's 
side,  thus  eliminating  glare  on  the  PC  display. 

The  selected  font  results  in  a  12  pixel  high  by  8  pixel  wide 
character.  At  24  inches  away,  the  image  plane  appears  6  inches  high,  which 
translates  into  40  pixels  per  inch.  The  net  result  is  a  character  height 
that  subtends  40  minutes  of  visual  angle,  which  exceeds  minimum 
requirements  (15  minutes)  by  a  large  margin. 
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SECTION  3 


PROCEDURE 

3.1  SEQUENCE  OF  EVALUATION  ACTIVITIES 

The  subjects  were  first  briefed  on  the  overall  purpose  of  the 
evaluation,  emphasizing  that  their  typing  ability  was  not  being  tested  and 
that  their  personal  opinions  of  the  Private  Eye  were  the  focus  of  the 
evaluation.  Each  subjects 's  job  title,  level  of  experience,  prior 
experience  with  a  keyboard,  vision  defects,  and  eye  dominance  were 
recorded.  (The  dominant  eye  is  the  one  preferred  for  aiming  a  gun  or  a 
camera,  analogous  to  right  or  left  handedness.)  Appendix  B  is  the  form 
used  for  recording  this  information  as  well  as  for  recording  the  subjective 
rating  data. 

Next,  the  subjects  were  shown  the  flight  strip  formats.  (These  flight 
strips  are  currently  used  in  air  traffic  control  to  post  current  data, 
providing  documentation  of  each  flight's  departure,  en  route,  and  arrival 
instructions.)  After  being  instructed  in  the  data  entry  procedures, 
subjects  were  asked  to  work  as  quickly  and  accurately  as  possible.  When 
using  the  Private  Eye,  they  were  instructed  to  put  it  in  front  of  their 
dominant  eye,  and  to  take  their  time  in  adjusting  it  into  the  most 
comfortable  position,  up/down,  left/right,  and  front/back. 

Key  entries,  on  a  stroke-by-stroke  basis,  were  displayed  on  the  PC's 
monitor  in  the  flight  strip  format  shown  in  figure  3-1.  Fields  1,  2,  3,  6, 
7,  8,  and  9  were  always  entered;  field  9a  required  data  entries  on  16  of 
the  22  flight  strips.  Fields  3  and  6  contained  only  numeric  characters, 
field  8  contained  only  alpha  characters,  and  the  remaining  fields  were 


8 


mixed;  only  fields  9  and  9a  contained  spaces.  All  characters  were  keyed 
without  regard  for  capitalization,  since  all  entries  were  displayed  in 
upper  case.  When  a  field  was  completed,  the  subject  checked  the  accuracy 
of  the  entry  and  hit  the  ENTER  key;  the  system  then  automatically  "tabbed" 
to  the  next  field.  When  all  fields  of  a  flight  strip  had  been  entered,  the 
subject  hit  the  ESCAPE  key  to  proceed  to  the  next  strip. 

Errors  detected  by  the  subjects  during  entry  of  a  field  were  corrected 
by  backspacing  and  reentry,  and  were  not  counted  as  errors  by  the  data 
collection  software.  If  the  errors  were  not  detected  before  hitting  ENTER, 
they  could  not  be  subsequently  corrected,  and  were  recorded. 

To  counterbalance  the  order  of  presentation,  nine  subjects  began  entry 
of  data  from  paper,  and  eight  subjects  began  data  entry  from  the  Private 
Eye.  When  all  22  flight  strips  had  been  entered,  the  subjects  switched  to 
the  alternate  presentation  mode  and  again  entered  the  same  22  strips  in  the 
same  order. 

After  the  flight  strips  had  been  entered  from  both  the  paper  and 
Private  Eye,  the  rating  scales  in  appendix  B  were  read  to  the  subjects  and 
each  rating  and  its  explanation  were  recorded  verbatim.  The  subjects 
required  about  20  minutes  to  enter  the  information  from  the  Private  Eye, 
about  20  minutes  to  enter  it  from  the  paper,  and  about  20  minutes  to 
perform  the  ratings.  Each  subject's  evaluation  session  thus  lasted  about 
an  hour. 


3.2  SUBJECT  CHARACTERISTICS 

The  primary  restriction  on  the  selection  of  subjects  was  that  they 
have  some  amount  of  technical  training,  since  it  is  likely  that  any  USAF 
application  of  the  Private  Eye  will  be  technical  in  nature;  for  this 
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reason,  purely  clerical  personnel  were  excluded.  Aside  from  this 
restriction,  an  effort  was  made  to  include  subjects  with  a  wide  range  of 
characteristics  in  terms  of  level  of  experience,  vision,  and  keyboard 
skills . 

A  total  of  17  subjects  were  used  in  this  evaluation,  none  of  whom  had 
any  previous  experience  with  the  Private  Eye.  Thirteen  of  them  were  MITRE 
technical  staff,  ranging  in  level  of  experience  from  recent  hires  to 
individuals  with  over  20  years  of  experience;  the  remaining  four  subjects 
were  either  technical  aides  or  co-op  students. 

Twelve  of  the  subjects  had  right  eye  dominance,  and  five  had  left  eye 
dominance.  (The  dominant  eye  was  used  to  view  the  Private  Eye.)  Nine  of 
the  subjects  wear  eyeglasses  or  contact  lenses,  which  were  always  worn 
throughout  the  study  session.  Of  these  nine  subjects,  one  required 
correction  only  for  close-up,  four  only  for  distance,  and  four  for  both 
close-up  and  distance;  four  of  the  nine  subjects  requiring  corrections  were 
also  corrected  for  astigmatism. 

Five  of  the  subjects  reported  typically  using  a  keyboard  from  1  to  8 
hours  per  week;  five  reported  using  a  keyboard  8  to  24  hours  per  week;  five 
reporting  using  a  keyboard  over  24  hours  per  week.  Two  subjects  very 
seldom  use  keyboards  and  were  quite  unfamiliar  with  them;  for  this  reason, 
their  data  was  not  included  in  the  key  input  results  reported  in  section 
4.2,  "Performance  Data." 
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SECTION  4 


RESULTS 


4.1  SUBJECTIVE  DATA 

The  first  few  questionnaire  items  requested  general  observations  and 
comments  from  the  subjects,  and  then  the  questionnaire  progressed  to  more 
specific  topics.  They  included  a  rating  scale,  which  required  the  subject 
to  quantify  his/her  opinion  in  whole  numbers  on  a  scale  from  1  to  7 .  More 
favorable  responses  were  given  smaller  numbers,  and  less  favorable 
responses  were  given  larger  numbers;  subjects  were  only  allowed  to  give 
whole  number  ratings.  After  each  rating  was  recorded,  the  subject  was 
asked  to  explain  why  the  particular  number  had  been  given. 

Ratings  considered  to  be  generally  favorable  are  1  or  2 ;  those 
considered  to  be  generally  neutral  range  between  3  and  5,  around  the 
scale's  midpoint  of  4;  unfavorable  ratings  are  6  or  7 .  Figure  4-1 
summarizes  the  ratings. 

Figure  4-1  summarizes  the  results  of  the  rating  scale  questions.  Each 
question  is  individually  listed  below,  along  with  the  average  rating  given 
by  all  17  subjects,  the  rating's  standard  deviation,  and  a  sample  of  the 
subjects'  rationales  for  the  ratings. 

o  "In  general,  how  did  you  like  the  Private  Eye?" 

A  rating  of  1  -  liked;  a  rating  of  7  -  disliked 

Mean  Score:  3.2 

Standard  Deviation:  1.1 
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Figure  4-1.  Summary  of  Subjective  Ratings 


As  was  expecCed,  there  were  a  variety  of  explanations  for  this  rating, 
most  of  which  were  detailed  in  subsequent  responses.  There  was  about  an 
even  mix  of  favorable  and  unfavorable  comments  made  by  the  subjects. 
Favorable  ones  included  a  reduced  number  of  required  eye  movements  and 
refocusing;  unfavorable  ones  concerned  difficulties  getting  used  to  the 
Private  Eye  (particularly  to  its  head  harness)  and  its  use  with  bifocals. 

o  "Please  rate  how  the  Private  Eye  affected  your  ability  to  enter 
data. " 

A  rating  of  1  -  helped  it;  a  rating  of  7  =  had  no  effect; 
a  rating  of  7  -  hurt  it 

Mean  Score:  3.4 
Standard  Deviation:  1.3 

The  respondents  were  generally  divided  about  the  Private  Eye's  support 
to  the  entry  of  flight  strip  data.  About  half  said  it  helped,  several 
stating  they  did  not  have  to  keep  moving  their  head  and  eyes  as  they  did 
when  using  the  printed  paper  source.  Others  said  that  using  paper  is  a 
more  familiar  task,  that  it  was  harder  to  keep  their  place  with  the  Private 
Eye . 


o  "Please  rate  how  easy  it  was  to  move  your  attention  between  the  PC 
Monitor  and  the  Private  Eye." 

A  rating  of  1  -  easy  to  do;  a  rating  of  7  -  hard  to  do 

Mean  Score:  3.0 
Standard  Deviation:  1.4 
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Host  of  the  subjects  reported  little  difficulty  in  looking  between  the 
PC  monitor  and  the  Private  Eye.  Of  the  few  who  had  problems,  one  said  that 
because  of  his  bifocals,  he  had  to  tilt  his  head  to  view  the  monitor. 

o  "Please  rate  how  easy  it  was  to  move  your  attention  between  the 
keyboard  and  the  Private  Eye . " 

A  rating  of  1  =  easy  to  do;  a  rating  of  7  =  hard  to  do 

Mean  Score:  2.6 

Standard  Deviation:  1.4 

Both  the  rating  scores  and  the  opinions  were  similar  to  the  previous 
question.  A  few  of  the  subjects  had  problems  looking  between  the  keyboard 
and  the  Private  Eye,  one  saying  that  the  Private  Eye  tended  to  obscure  his 
vision  of  the  keyboard.  Problems  were  again  reported  when  wearing 
bifocals.  It  is  likely  that  the  subjects'  typing  ability  affected  both 
questions  3  and  4,  since  a  more  skilled  typist  would  tend  to  look  between 
the  Private  Eye  and  the  monitor  or  keyboard  fewer  times  than  would  a  less 
skilled  typist.  The  one  reported  problem  with  the  keyboard's  being 
obscured  might  have  been  solved  by  slightly  raising  the  Private  Eye;  thus, 
these  two  questions  could  also  have  been  affected  by  the  positioning  of  the 
Private  Eye. 

o  "Please  rate  the  color  of  the  characters  on  the  Private  Eye." 

A  rating  of  1  -  likes;  a  rating  of  7  -  disliked 

Mean  Score:  2.8 

Standard  Deviation:  1.2 
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About  half  of  the  subjects  were  noncommittal  or  indifferent  about  the 
Private  Eye's  red-on-black  display.  Of  those  with  an  opinion,  four  held 
positive  attitudes  toward  the  color,  saying,  for  example,  that  the  color 
did  not  interfere  with  the  color  of  the  monitor.  Four  subjects  had 
negative  opinions,  preferring  white,  amber,  or  green  characters.  Based 
upon  the  subjects'  subsequent  explanations,  it  is  likely  that  several  of 
the  ratings  were  not  based  solely  upon  color,  but  were  also  (positively) 
affected  by  the  brightness  and  contrast  of  the  Private  Eye  display. 

o  "Please  rate  the  legibility  of  the  characters  on  the  Private  Eye. 

A  rating  of  1  -  legible;  a  rating  of  7  -  illegible 

Mean  Score:  1.9 

Standard  Deviation:  0.9 

Although  the  legibility  ratings  were  relatively  high,  there  were 
complaints  from  many  subjects  that  the  spacing  between  adjacent  words  was 
too  small.  Several  subjects  said  that  it  took  a  moment  to  focus  their  eye 
on  the  Private  Eye,  whereupon  the  legibility  became  acceptable.  There  was 
also  confusion  reported  between  the  letters  "U"  vs.  "V",  "C"  vs.  "G",  and 
between  the  letter  "1"  vs.  the  numeral  "1".  Character  legibility  is  a 
function  of  the  brightness  and  contrast  of  the  characters,  as  well  as  the 
font  style  of  the  characters  themselves.  It  is  likely  that  the  Private 
Eye's  relatively  high  degree  of  brightness  and  contrast  compensated  for 
perceived  deficiencies  in  the  fonts,  thus  accounting  for  the  generally 
positive  ratings  given  to  this  question. 

o  "Please  rate  your  fatigue  during  and  after  using  the  Private  Eye. 

A  rating  of  1  -  little  fatigue;  a  rating  of  7  -  much  fatigue 

Mean  Score:  2.3 

Standard  Deviation:  1.3 
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About  1/3  of  the  subjects  reported  some  degree  of  fatigue  after 
completing  the  study.  Four  of  the  subjects  reported  some  eye  fatigue,  but 
it  only  ranged  from  slight  to  moderate;  two  others  reported  neck  fatigue  or 
discomfort  related  to  wearing  the  head  harness.  Although  the  task  was 
designed  to  require  a  rather  unrealistically  concentrated  use  of  the 
Private  Eye,  it  only  lasted  for  about  20  minutes.  If,  as  intended,  the 
concentrated  nature  of  this  task  compensated  for  its  brief  duration,  it  is 
possible  that  these  ratings  could  be  representative  of  a  longer,  more 
appropriate  task. 

o  "Please  rate  the  comfort  of  wearing  the  Private  Eye." 

A  rating  of  1  -  comfortable;  a  rating  of  7  -  uncomfortable 

Mean  Score:  2.9 

Standard  Deviation:  1.9 

As  noted  above,  ample  time  was  given  for  the  subjects  to  adjust  the 
position  of  the  Private  Eye  and  its  head  harness.  In  spite  of  this,  about 
half  of  the  subjects  had  adverse,  although  not  severe,  comments.  Most 
complained  about  the  unbalanced  headset  which  placed  more  pressure  on  one 
side  of  the  head;  one  suggested  some  kind  of  counterbalance. 

o  "Did  you  notice  flicker  on  the  Private  Eye?  If  yes,  please  rate 
how  bothersome  it  was." 

Five  subjects  noticed  flicker,  ghosting,  or  character  movement  on  the 
Private  Eye.  Three  perceived  some  movement  of  the  Private  Eye  characters 
themselves,  and  a  fourth  felt  the  movement  was  due  to  refocusing  his  eye, 
not  to  the  Private  Eye.  Also,  two  of  these  five  subjects  sometimes 
perceived  ghosts  or  after-images  on  the  display.  The  subjects  rated  their 
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perception  of  perceived  flicker  and  its  related  phenomena  from  "not 
bothersome"  (a  rating  of  1)  to  "very  bothersome"  (a  rating  of  7).  The  mean 
rating  by  these  five  subjects  was  3.5  with  a  standard  deviation  of  2.1. 

o  "Did  you  have  any  problems  judging  how  far  away  objects  were  from 
you  (that  is,  your  ability  to  judge  close-up  distances)  when  you 
were  using  the  Private  Eye?  If  yes,  please  describe  them.  When 
did  they  occur?  Is  there  any  way  you  can  think  of  to  deal  with 
them? " 

This  question  was  intended  to  establish  whether  blocking  one  eye  (and 
the  resultant  lack  of  "binocular  parallax")  impeded  the  subject’s  depth 
perception,  and  thus  his/her  task  performance.  Only  1  of  the  17  subjects 
said  he  had  a  depth  perception  problem,  and  even  this  was  relatively  mild 
since  he  was  able  to  compensate  for  it.  Several  '  .  c^s  noted  that  they 
were  able  to  look  below  the  Private  Eye  and  thus  retain  normal  vision  of 
the  keyboard.  Thus,  the  subjects  were  either  able  to  appropriately  locate 
the  Private  Eye  or  to  use  other  depth  pej.oention  cues  than  binocular 
parallax;  the  result  was  virtually  no  interference  with  depth  perception. 

o  "Did  you  develop  any  special  techniques  for  doing  the  task  while 
you  were  wearing  the  Private  Eye?" 

This  question  was  intended  to  enlarge  upon  any  other  special 
compensations  that  the  subjects  developed  to  deal  with  perceived  display 
problems.  About  one  third  responded  that  they  had  developed  some  special 
"tricks"  to  help  them  perform  the  task,  but  most  did  not  attempt  any 
special  techniques.  Several  noted  that  they  carefully  positioned  the 
Private  Eye  so  that  they  could  more  easily  look  at  the  monitor  and/or  the 
keyboard;  one  noted  that  he  attempted  to  use  only  eye  movements,  not  head 
movements,  when  looking  at  the  keyboard.  All  three  bifocal  users  found 
that  using  the  top  of  the  lens  (the  distance  correction)  was  more  suitable. 
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o  "Do  you  have  any  other  comments  or  suggestions  about  the  Private 
Eye?  Technical  features?  Application  to  this  type  of  task? 
Application  to  any  other  type  of  task?" 

Regarding  the  features  of  the  Private  Eye,  a  shortfall  noted  by  two 
subjects  was  that  field  8,  the  rightmost  field,  could  not  be  properly 
displayed  on  the  Private  Eye  and  that  the  display  should  be  wider;  however, 
since  only  two  reported  this  problem,  perhaps  the  problem  was  caused  by 
improper  placement  of  the  device.  Another  subject  reiterated  his  problems 
using  bifocals;  a  third  complained  that  the  Private  Eye  blocked  part  of  the 
information  on  the  PC  monitor.  One  subject  suggested  that  the  typed  input 
be  "echoed"  on  the  bottom  line  of  the  Private  Eye  as  it  is  entered;  another 
suggested  that  the  brightness  of  the  Private  Eye's  display  be  adjustable;  a 
third  said  it  should  be  capable  of  being  flipped  up,  moving  it  out  of  the 
line  of  sight. 

As  expected,  it  was  noted  that  the  Private  Eye  is  not  suited  to  a 
concentrated  data  input  task,  particularly  over  an  extended  period  of  time; 
the  paper  was  said  to  be  an  easier  source  of  data  for  such  a  task. 

A  suggested  application  was  a  task  in  which  the  user  is  not  able  to 
stay  in  front  of  a  fixed  display.  Another  was  a  suggestion  for  an 
application  requiring  the  display  of  slides  or  graphic  data  which  change 
infrequently  and/or  would  not  have  to  be  continuously  monitored.  One 
subject  suggested  the  ability  to  see  through  the  Private  Eye,  past  the 
display  information,  similar  to  an  aircraft's  heads-up  display.  An 
application  was  also  suggested  in  which  information  must  be  displayed  to  an 
individual  in  a  darkened  environment  (e.g.,  on  an  instructor's  display  in  a 
training  simulator) . 
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One  subject  first  used  his  nondominant  eye  for  the  entire  procedure, 
and  subsequently  repeated  the  Private  Eye  portion  of  the  evaluation  using 
his  dominant  eye.  He  reported  that  using  the  dominant  eye  seemed  less 
irritating  and  made  it  easier  to  concentrate  on  the  display  with  both  eyes 
open.  Although  he  perceived  jitter  during  both  sessions,  it  seemed  less 
aggravating  with  the  dominant  eye. 


4 . 2  PERFORMANCE  DATA 

The  objective  performance  data  was  quantified  in  terms  of  the  average 
number  of  characters  per  minute  and  number  of  incorrect  fields  per  flight 
strip.  It  will  be  recalled  that  errors  were  not  counted  which  were 
corrected  by  the  subject  before  the  field  was  entered;  only  uncorrected 
errors  were  recorded. 

Figure  4-2  shows  the  average  number  of  character/minute  entered  on 
each  of  the  22  successive  flight  strips,  including  the  time  to  detect  and 
correct  errors.  The  figure  indicates  that  the  average  input  rate  for  paper 
was  higher  then  for  the  Private  Eye  on  nearly  every  flight  strip.  Of  more 
interest  is  the  additional  indication  from  this  figure  that  performance  on 
both  printed  material  and  Private  Eye  depended  to  a  great  extent  on  the 
particular  strip  being  entered.  For  example,  the  rate  of  entry  from  both 
sources  was  lower  on  flight  strip  7  and  19  and  relatively  higher  on  strips 
9  through  13. 

The  Private  Eye  had  an  average  input  rate  of  76.6  characters/minute 
for  all  flight  strips  with  a  standard  deviation  of  26.2;  the  overall  rate 
was  80.1  characters/minute  for  the  paper  material  with  a  standard  deviation 
of  22.7.  Because  of  this  substantial  variability  under  both  conditions, 
this  3.5  character/minute  difference  between  the  means  of  the  two  sources 
was  not  statistically  significant. 
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Figure  4-2.  Key  Entry  Rates  from  Private  Eye  and  from  Paper 


Of  the  roughly  22,000  keystrokes  made  by  all  subjects  under  each 
condition,  there  was  a  total  of  95  errors  when  using  the  Private  Eye  and  a 
total  of  70  errors  when  using  the  paper.  Most  of  the  errors  in  both 
conditions  involved  hitting  an  incorrect  character;  the  next  most  frequent 
error  was  the  insertion  of  an  additional  character;  the  third  most  frequent 
error  was  skipping  a  character.  Apparently  the  flight  strips  were  roughly 
equivalent  in  difficulty  since  no  flight  strips  appeared  to  be  consistently 
more  error  prone  than  others . 


4.3  RELATIONSHIP  TO  SUBJECTS'  CHARACTERISTICS 

Subjects  were  divided  into  two  groups,  those  who  gave  more  favorable 
ratings  (three  or  more  questions  with  ratings  of  1)  and  those  who  gave  less 
favorable  ratings  (three  or  more  questions  with  ratings  of  5  or  above). 
There  was  no  apparent  relationship  between  subjects'  tendency  to  give 
favorable  or  unfavorable  ratings  and  their  needs  for  corrective  lenses  nor 
their  amount  of  reported  keyboard  experience. 
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SECTION  5 


DISCUSSION 


5.1  SUBJECTIVE  DATA,  OVERALL 

It  is  widely  accepted  that  rating  scales  are  susceptible  to  several 
biases  due  to  people's  "built-in"  attitudes  and  characteristics.  One  of 
these  biases  which  may  have  been  a  factor  in  this  evaluation  is  called  the 
bias  of  "central  tendency"  or  the  predisposition  to  avoid  extreme 
statements,  positive  or  negative,  tending  to  force  ratings  toward  the 
midpoint  of  the  scale. 

The  subjects'  overall  ratings  of  the  Private  Eye  ranged  from  slightly 
favorable  to  neutral,  as  indicated  by  their  ratings  and  their  stated 
opinions.  Many  of  the  ratings  were  toward  the  positive  end  of  the  rating 
scale,  falling  mostly  between  2.8  and  4.2.  The  first  question  indicated  a 
generally  neutral  attitude  toward  the  device.  More  detailed  questions 
indicated  slightly  positive  attitudes  toward  the  Private  Eye's  legibility 
and  color  and  more  neutral  attitudes  for  the  suitability  of  the  Private  Eye 
to  the  task,  since  the  application  was  not  optimized. 


5.2  SUBJECTIVE  DATA,  INDIVIDUAL  ISSUES 

Depth  Perception.  The  Private  Eye  blocks  the  visual  field  of  one  eye, 
thus  limiting  binocular  parallax  cues  to  stereoscopic  vision.  In  spite  of 
this  blockage,  it  did  not  impede  the  task  chosen  for  this  study.  There  was 
not  a  substantial  requirement  for  stereopsis  when  looking  at  the  keyboard 
or  PC  monitor,  and  it  is  likely  that  single  eye  vision  would  suffice  even 
if  the  subjects  had  not  learned  to  look  around  the  Private  Eye.  However, 
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other  applications  may  depend  upon  binocular  parallax  to  a  greater  extent. 
If  these  applications  also  limit  other  cues,  stereoscopic  perception  may  be 
reduced.  Each  application  of  the  device  thus  must  be  evaluated  with  this 
limitation  in  mind,  particularly  if  it  involves  a  safety  risk.  To  retain 
binocular  parallax  cues  while  keeping  the  Private  Eye  conveniently 
available,  it  is  suggested  that  the  device  be  able  to  be  flipped  up, 
totally  out  of  the  line  of  sight. 

Color /Contrast .  The  red  color  of  the  display  did  not  cause  consistent 
objections  from  the  subjects,  although  some  preferred  other  colors  perhaps 
because  they  are  accustomed  to  them.  The  contrast  and  brightness  of  the 
display  seemed  quite  good,  but  a  possible  enhancement  would  be  to  make  the 
brightness  adjustable. 

Fatigue .  There  seems  to  be  little  reported  fatigue  in  the  eye  used  to 
view  the  Private  Eye.  Although  the  Private  Eye  was  only  used  for  about  20 
minutes,  this  time  limitation  might  be  somewhat  offset  by  the  concentrated 
nature  of  the  task.  These  results  might  thus  be  generalized  to  a  longer 
but  less  concentrated  task;  substantial  eye  fatigue  may  not  occur  up  to  a 
period  of  several  hours  if  the  Private  Eye  is  not  continuously  viewed. 

Most  of  the  fatigue  seemed  to  result  from  the  head  harness  and/or  the  need 
to  keep  the  head  steady  while  viewing  the  Private  Eye,  in  spite  of  the  fact 
that  the  device  was  worn  for  only  a  short  time.  The  vendor  has  redesigned 
this  harness,  and  the  new  design  may  compensate  for  some  of  these  reported 
problems . 

Display  Artifacts.  A  few  of  the  subjects  noticed  some  vibration  in 
the  device,  or  some  movement  or  "ghosting"  of  the  images,  but  these 
subjects  rated  the  perceived  effects  as  not  particularly  objectionable. 
Perception  of  some  artifacts  by  a  minority  of  observers  is  not  an  unusual 
occurrance  in  display  evaluations.  Since  these  phenomena  were  not 
consistently  reported  by  all  subjects,  they  might  be  due  to  some  individual 
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characteristic  of  the  subject  or  perhaps  to  some  other  display  artifact. 

In  a  conversation  with  E.  Peli  (Tufts  and  Harvard  Universities),  he  noted  a 
similar,  transient  artifact  with  the  Private  Eye  when  subjects  moved  their 
heads.  The  track  in  the  present  study  did  not  require  head  or  eye 
movement,  but  it  is  possible  that  these  few  subjects  tended  to  move  around 
more  than  the  others,  thus  accounting  for  the  reported  phenomena.  It  is 
suggested  that  these  effects  be  investigated  further. 

Font .  The  character  font  for  the  display  must  be  carefully  chosen. 
Differences  in  pixel  aspect  ratios  result  in  slight  changes  in  the  height/ 
width  ratio  of  characters  when  displayed  on  the  Private  Eye.  For  example, 
standard  PC  monitor  text  appears  smaller  on  the  Private  Eye.  Although  a 
new  font  was  designed  for  this  experiment  (18  point  Times  font),  the  result 
was  tall  thin  characters  and  narrow  blanks  between  words.  Although  zeros 
and  the  "Os"  were  deleted  from  the  study,  confusion  related  to  the  font 
design  still  occurred  fairly  frequently.  It  is  suggested  that  font(s)  be 
specifically  designed  for  the  Private  Eye  to  overcome  this  limitation 
(e.g.,  Lincoln-MITRE  font). 

Device  Adjustment.  The  Private  Eye  must  be  very  carefully  adjusted, 
and  the  users  must  be  carefully  instructed  in  how  to  best  perform  these 
adjustments.  They  must  be  given  both  the  time  and  information  to  locate 
the  head  harness  and  the  display  in  the  most  comfortable  and  workable 
position.  It  is  possible  that  the  subject  who  reported  difficulty  in 
seeing  all  of  the  Private  Eye  did  not  have  the  device  positioned  close 
enough  to  his  eye.  Similarly,  those  bifocal  wearers  who  experienced 
difficulties  might  have  avoided  their  problems  had  they  positioned  the 
device  and/or  adjusted  the  focus  to  allow  them  to  consistently  use  either 
the  top  of  their  glasses  (with  the  device  focused  for  distance  viewing)  or 
the  bottom  of  their  glasses  (with  the  device  focused  for  close  up)  .  Ease 
of  adjustment  should  be  a  prime  concern  in  the  redesign  of  the  head 
harness . 
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5.3  PERFORMANCE  DATA 


There  was  not  a  substantial  difference  in  the  subjects'  performance 
between  entering  data  from  the  Private  Eye  and  entering  it  from  the  paper 
source  material,  although  the  average  performance  was  usually  poorer  when 
using  the  Private  Eye.  Even  if  the  input  rate  had  been  statistically 
different  under  the  two  conditions,  the  obtained  difference  of  only  3.5 
character/minute  would  be  trivial  for  nearly  any  technical  application  of 
the  device.  Likewise,  when  comparing  errors  under  the  two  conditions,  a 
total  difference  of  15  errors  by  all  17  subjects  entering  the  total  amount 
of  information  on  all  22  flight  strips  could  be  considered  to  be  a 
relatively  trivial  difference. 

It  is  possible  that  there  are  some  learning  effects  in  using  the 
Private  Eye.  Subjects  might  have  become  more  used  to  the  device  and  more 
proficient  with  it  had  they  used  it  for  longer  periods  of  time  over  several 
days.  However,  this  evaluation  was  intended  to  be  relatively  brief  and  to 
use  as  many  subjects  as  possible.  A  systematic  examination  of  learning  was 
thus  not  feasible. 


5.4  GENERAL  CONCLUSION 


The  Private  Eye  was  slightly  slower  and  more  error  prone  than  was  the 
paper  material  in  this  simulated  air  traffic  control  task.  It  is  thus  not 
an  optimum  device  for  continuous,  concentrated  data  entry.  However,  in 
spite  of  its  performance  restrictions,  subjects  generally  felt  it  was 
acceptable . 
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To  summarize,  this  evaluation  identified  no  intrinsic  limitations  in 
the  usability  of  the  Private  Eye.  Possible  applications  should  capitalize 
on  the  portable  nature  of  the  display  and  its  privacy  aspects,  displaying 
data  not  requiring  continuous  reading.  Attention  must  also  be  given  to  the 
appropriate  choice  of  an  input  device  (e.g.,  voice  recognition).  Given 
such  applications,  this  type  of  wearable  display  could  have  significant 
advantages . 
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The  following  were  used  as  stimulus  material  from  both  the  Private  Eye  and  the 
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Appendix  B 


Name  : _ 

Interviewer: 


BACKGROUND  DATA 

Subject's  job  title  (e.g.,  MTS,  secretary  etc): _ 

Year  in  which  Bachelors  degree  was  received: _ 

Subject's  dominant  eye:  _ right  _ left 

Subject's  eye  used  for  Private  Eye:  _ right  _ left 

Subject's  vision: 

Normally  wears  glasses: _ yes _ no 

Normally  wears  contact  lenses:  _ yes _ no 

If  yes,  glasses/contact  lenses  needed  for: 

close  up _ 

distance _ 

both  (bifocals) _ 

astigmatism _ 

Glasses/contact  lenses  worn  during  study:  _ yes _ no 

Subject's  estimated  hours  per  week  that  he/she  uses  any  type  of  keyboard 
(typewriter,  PC,  etc.): 

_ less  than  1,  _ 1-2,  _ 3-4,  _ 4-8, 

_ _ 8-12,  _  12-16,  _  16-24,  24-30,  _  over  30 


(rev  2.5) 

Date: _ 

Start  Time: 
Stop  Time: 


OBSERVER'S  COMMENTS 

The  observer  should  note  below  any  of  the  subject's  comments  (verbatim) 
or  any  observations  of  behaviors  which  are  relevant  to  his/her  opinion  or 
use  of  the  Private  Eye. 
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INTERVIEW 

Interview  Procedure:  Interviewer  reads  each  question  to  subject,  asks 
him/her  to  make  a  rating  (1  of  7  numbers  only ),  and  then  asks  why  that 
rating  was  given. 


1)  In  general,  how  did  you  like  the  Private  Eye. 


1  2  3  4  5  6  7 

Liked  Disliked 


Please  explain 


2)  Please  rate  how  the  Private  Eye  affected  your  ability  tP  enter  data 


1  2 
Helped  it 


3  4  5 

Had  no  effect 


6  7 

Hurt  it 


Please  explain: 
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3)  Please  rate  how  easy  it  was  to  move  your  attention  between  the  PC 
Monitor  and  the  Private  Eye. 


1  2  3  4  5  6 

Easy  to  do 

Please  explain: 


_ l 

7 

Hard  to  do 


4)  Please  rate  how  easy  it  was  to  move  your  attention  between  the 
keyboard  and  the  Private  Eye. 


1  2  3  4  5  6  7 

Easy  to  do  Hard  to  do 

Please  explain: 


5)  Please  rate  the  color  of  the  characters  on  the  Private  Eye 


1  2 
Liked 

Please  explain. 


3  4  5  6  7 

Disliked 
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6)  Please  rate  the  legibility  of  the  characters  on  the  Private  Eye 


1  2  3  4  5  6  7 

Legible  Illegible 

Please  explain. 


7)  Please  rate  vour  fatigue  during  and  after  using  the  Private  Eye 

1 _ I _ I _ I _ I _ I _ I _ l 

1  2  3  4  5  6  7 

Little  fatigue  Much  fatigue 

Please  explain: 


8)  Did  you  notice  flicker  on  the  Private  Eye?  If  yes,  please  rate  how 
bothersome  it  was. 


1  2  3  4  5  6  7 

Not  bothersome  Very  bothersome 

Please  explain: 
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9)  Please  rate  the  comfort  of  wearing  the  Private  Eye 


1  2  3  4  5  6  7 

Comfortable  Uncomfortable 

Please  explain: 


10)  Did  you  have  any  problems  judging  how  far  away  objects  were  from 
you  (that  is,  your  ability  to  judge  close-up  distances)  when  you  were  using 
the  Private  Eye? 

-If  yes,  please  describe  them: 


-When  did  they  occur? 


-Is  there  way  you  can  think  of  to  deal  with  them? 


11)  Did  you  develop  any  special  techniques  for  doing  the  task  while  you 
were  wearing  the  Private  Eye? 
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12)  Do  you  have  any  other  comments  or  suggestions  about  the  Private  Eye? 
-Technical  features? 

-Application  to  this  type  of  task? 

-Application  to  any  other  type  of  task? 


46 


